Intelligent coding metasurface is a kind of information-carrying metasurface that can manipulate electromagnetic waves and associate digital information simultaneously in a smart way. One of its widely explored applications is to develop advanced schemes of dynamic holographic imaging. By now, the controlling coding sequences of the metasurface are usually designed by performing iterative approaches, including the Gerchberg–Saxton (GS) algorithm and stochastic optimization algorithm, which set a large barrier on the deployment of the intelligent coding metasurface in many practical scenarios with strong demands on high efficiency and capability. Here, we propose an efficient non-iterative algorithm for designing intelligent coding metasurface holograms in the context of unsupervised conditional generative adversarial networks (cGANs), which is referred to as physics-driven variational auto-encoder (VAE) cGAN (VAE-cGAN). Sharply different from the conventional cGAN with a harsh requirement on a large amount of manual-marked training data, the proposed VAE-cGAN behaves in a physics-driving way and thus can fundamentally remove the difficulties in the conventional cGAN. Specifically, the physical operation mechanism between the electric-field distribution and metasurface is introduced to model the VAE decoding module of the developed VAE-cGAN. Selected simulation and experimental results have been provided to demonstrate the state-of-the-art reliability and high efficiency of our VAE-cGAN. It could be faithfully expected that smart holograms could be developed by deploying our VAE-cGAN on neural network chips, finding more valuable applications in communication, microscopy, and so on.